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DETAILED ACTION 

Response to Arguments 

1 . Applicant's arguments with respect to claims 1-8, 10-34, and 36-40 have been 
considered but are moot in view of the new ground(s) of rejection. 

Claim Rejections - 35 USC § 103 

2. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

3. Claim 1-8, 10-30, 36, 39, and 40 rejected under 35 U.S.C. 103(a) as being 
unpatentable overZawilinski, U.S. Patent No. 5,676,138, in view of Hoffberg et al., U.S. 
Patent No. 6,400,996. 

Regarding claim 1 and 40, Zawilinski discloses a multimedia system comprising 
an emotional response analyzer. The system further comprises sensors for 
sensing the user's physical reactions (i.e., gaze, galvanic skin response, etc.) to 
various stimtrli, including commercials, wherein said stimuli are associated with 
recognizable emotional responses (i.e., surprise, anger, disgust, etc.). Moreover, 
this information can be indexed and associated with whatever stimuli were being 
displayed at the time of said response. (Fig. 1 & 2; Abstract; Col. 1, Ln. 6-47; Col. 
3, Ln. 39-Col. 4, Ln. 47; Col. 5, Ln. 34-43, 63-Col. 6, Ln. 26, 35-Col. 7, Ln. 5; Col. 
9, Ln. 50-64). But, Zawilinski fails to disclose a microphone picking up 
vocalizations made by the user. In an analogous art, however, Hoffberg discloses 
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using biometric sensor data to create a dynamic user preference profile, wherein 
voice patterns of the user are detected and associated with a recognizable 
emotional response (i.e., moods; col. 116, 11. 3-14, col. 61, 11. 14-30, col. 65, 
U. 23-55; a microphone, or electro-acoustic transducer, is inherently present in 
where an acoustic signal (i.e., voice) is interpreted by an electrical device). Thus, 
it would have been obvious to one having ordinary skill in this art at the time of 
Applicant's invention to modify the at least one sensor of Zawilinski to include a 
microphone for picking up vocalizations, as taught by Hoffberg, to provide 
additional physiological data from which the user's preferences can be inferred. 
Furthermore, Hoffberg discloses of a recommender that is configured to 
recommend subsequent programs based on the viewer preferences (col. 65 
lines 16-22; The system provides an intelligent, adaptive pattern 
recognition function in order to recommend programs to the user based on 
the user input, a past history of use, and a context of use.) Therefore it 
would have been obvious to one of ordinary skill in the art to combine the 
teachings of Zawilinski and Hoffberg to provide recommendations to the 
user based on viewer preferences. 

Claims 34 and 39 are encompassed within the rejection of Claim 1. Thus, it 
is analyzed and rejected as previously discussed. 
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As to claim 2, Zawilinski and Hoffberg together disclose the claimed memory 
device (see Hoffberg, col. 1 13, U. 47-66). 

As to claim 3, Zawilinski further discloses the system comprises a plurality of 
sensors. (Col. 6, Ln. 3-26). 

As to claim 4, Zawilinski further discloses the claimed sensor-signal receiver 
(rigA, item 18; col. 6, 11. 27-35). 

As to claim 5, Zawilinski further teaches the system can analyze various 
physiological data including heart rate, galvanic skin response, etc. (i.e., 
aggregation of signals; see Zawilinski as applied to claim 1, above). 

As to claims 6-8, Zawilinski and Hoffberg together disclose the use of a camera 
(video pattern recognition; Hoffberg, col. 1 16, 1 1 .3-14) and video recorder 
recording images captured by the camera to an image library for comparing to 
video images received from the camera (internal database; Hoffberg, col. 59, 1 1 . 
33-61). 

As to claim 10, Hoffberg, further discloses the system comprises an 
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environmental sensor for sensing a change in the viewing environment, as 
claimed (col. 127, U. 5-40). 

Regarding claim 11, Zawilinski and Hoffberg together disclose the claimed 
subject matter. (See the rejection of claim 1; Hoffberg discloses 
determining when a program segment is being received that corresponds 
to a pre-selected viewer response previously associated with a physical 
condition status (i.e., user's mood; col. 116, 11. 3-14, 33-49, col. 118, 11. 27- 
43).) 

As to claim 12, Zawfilinski and Hoffberg together disclose monitoring a plurality of 
viewer physical conditions (Zawilinski, col. 6, 1 1 .3-26). 

As to claim 13, Zawilinski further teaches the system can plot various stimuli 
changes overtime (i.e., changes in condition relative to a baseline level). (Col. 9, 
Ln. 12-20). 

As to claim 14, Zawilinski and Hoffberg together disclose the claimed subject 
matter. In particular, Hoffberg discloses the physical condition is body 
temperature (col. 61, n. 13-30). 
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As to claim 15, Zawilinski further teaches the system can analyzed the heart rate 
of a user, (see rejection of claim 1). 

As to claims 16 and 17, see the rejection of claims 6-8, above. 

As to Claim 18, Zawilinski and Hoffberg together disclose the claimed subject 
matter. In particular, Hoffberg discloses determining characteristics of a 
displayed programming segment, associating a viewer response corresponding 
to a physical condition with a viewer preference level, and applying the 
preference level to enhance program selection (col. 100, 11. 2-51 , col. 1 16, 1 1 . 
33-49, col. 118, 11. 21-43). 

As to claim 19, Hoffberg further teaches providing a notification that specified 
future programming will contain at least one segment possessing the at least one 
distinguishing characteristic (col. 116, 1. 50 - col. 117, 1. 5). 

As to claim 20, Hoffberg further teaches enhancing the program selection by 
inserting a segment possessing the distinguishing characteristic (col. 114, 11.12- 
23). 



As to claims 21 and 22, Hoffberg further discloses the characteristic is derived 
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from EPG information provided with the programming, including text (col. 150, U. 
38-54). 

As to claim 23, Hoffberg further discloses the claimed recorder for automatically 
recording the segment corresponding to a pre-selected viewer response (see 
portions of Hoffberg cited above). 

As to claim 24, Hoffberg further discloses extracting information related to the 
program segment that corresponds to pre-selected viewer response from the 
programming, and automatically displaying the information (col. 124, 11. 45-57). 

The limitations of claim 25 are encompassed within the portions of Zawilinski 
used to reject claim 1 . 

As to claim 26, Zawilinski further teaches the system can analyzed the galvanic 
skin response of a user. (Col. 6, Ln. 10-19). 

As to claims 27 and 28, Zawilinski further teaches the system is capable of 
measuring the gaze of the viewer and the electrical activity in muscles, which 
relates to negative/positive facial expressions (i.e., visually observable 
response). (Col. 4, Ln. 13-31 and citations of Claim 1). 
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As to Claim 29, Zawilinski further teaches the system can track the direction of 
the user's gaze. (Col. 9, Ln. 50-64). 

As to claim 30, Zawilinski further discloses the system is capable of analyzing the 
movement (i.e., changes in direction) of the viewers gaze and correlating the 
gaze direction with specific time periods (col. 9, 1 1 . 50-64, col. 4, II. 42-47). 

As to claim 36, Zawilinski and Hoffberg together disclose the method of claim 34, 
wherein an audible response is detected to infer the listener's mood. Official 
notice is taken that laughter is a well-known audible indication of mood. Thus, it 
would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the system of Zawilinski and Hoffberg to include listener 
laughter as an audible response, thereby providing an addition indication of the 
listener's mood. 

1 . Claims 31 and 32 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Zawilinski in view of Hoffberg applied to claim 27 above, and further in view of Black et 
al. (US Patent No. 5,774,591). 

Claim 31 recites the method of claim 27, wherein the visually observable 
response includes the furrowing of the viewer's brow. As discussed above, the 
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combined systems of Zawilinski and Hoffberg render obvious all limitations of 
Claim 27, and Zawilmski further teaches the system can analyzed the electrical 
impulses in the user's muscles (i.e., which can related to facial expressions), but 
fails to specifically recite analyzing eyebrow furrowing. However, within the same 
field of endeavor, Black et al disclose a similar system which analyzes changes 
in the viewer's eye brows. (Abstract; Col. 7, Ln. 1-45; Col. 26, Ln. 50-67; Col. 28, 
Ln. 28-62). Accordingly, it would have been obvious to one having ordinary skill 
in this art at the time of Applicant's invention to combine the systems of 
Zawilinski, Hoffberg, and Black in order to provide a system capable of analyzing 
facial features, thereby affecting the operation of a computer system. 

The limitations of claim 32 would be obvious variants of the limitations of claim 
31 . Since Black et al allow for the analysis of user eyebrows, analyzing the depth 
of movement in said eyebrows would only be an obvious variant. Accordingly, it 
would have been obvious to one having ordinary skill in this art at the time of 
Applicant's invention to modify the combined systems of Zawilinski, Hoffberg and 
Black in order to provide a system which analyzes the depth of furrows, thus 
allowing for an analysis of the level of a viewer facial response (i.e., level of 
anger, surprise, understanding, etc). 

2. Claims 33 and 38 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Zawilinski, in view of Hoffberg, as applied to claims 1 1 and 34 above, and further in 
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view of Bentolila etal. (US Pat. Pub. No. 2003/0101449). 

Zawilinski and Hoffberg together disclose the limitations of claims 1 1 and 34, but 
fail to discuss the limitations of claims 33 and 38. However, within the same field 
of endeavor, Bentolila et al disclose a similar system which utilizes a Hidden 
Markov technique. (Par. [0021]). Accordingly, it would have been obvious to one 
having ordinary skill in this art at the time of Applicant's invention to combine the 
systems of Zawilinski, Hoffberg and Bentolila in order to provide a system for 
user profile data prediction. 

3. Claim 37 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Zawilinski in view of Hoffberg, as discussed under claim 34, and further in view of 
Shinohara. (US Pat. Pub. No. 2003/0005431). 

Claim 37 recite the method of Claim 34, wherein the audibly observable 
response is the inflection (i.e., changes in pitch or tone) of a listener's 
vocalization, tending to indicate a question has been enunciated. As discussed 
above, the combined systems of Zawilinski and Hoffberg render obvious all 
limitations of Claim 34, but fail to specifically recite the limitations of Claim 37. 
However, within the same field of endeavor, Shinohara discloses a similar 
system which analyzes spectral data related to speech patterns, such as pitch, 
tone, frequency, etc., in order to identify a television viewer. (Par. [0026]). 
Accordingly, it would have been obvious to one having ordinary skill in this art at 
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the time of Applicant's invention to combine the systems of Zawilinski, Hoffberg, 
and Shinohara in order to provide a system which is capable of analyzing various 
tones, pitches, etc. of a given voice. 

Conclusion 

4. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to SAHAR A. BAIG whose telephone number is (571)270- 
3005. The examiner can normally be reached on 4/5/9. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Chris Kelley can be reached on 571-272-7331. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Chris Kelley/ 

Supervisory Patent Examiner, Art 
Unit 2623 
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